Basis Function Adaptation in Temporal Difference Reinforcement Learning

We examine methods for on-line optimization of the basis function for temporal difference Reinforcement Learning algorithms. We concentrate on architectures with a linear parameterization of the value function. Our methods optimize the weights of the network while simultaneously adapting the parameters of the basis functions in order to decrease the Bellman approximation error. A gradient-based method and the Cross Entropy method are applied to the basis function adaptation problem. The performance of the proposed algorithms is evaluated and compared using simulation experiments. keywords: Reinforcement Learning, Temporal difference algorithm, Cross Entropy method, Radial Basis functions.